Exploring Weights

Here we train the LNN on various task and see how the weights are



In [1]:

    
import sys
import os
sys.path.insert(0,'..')
sys.path.insert(0,'../layeredneuralnetwork/')

Let us create an LNN with 2 dimension input.



In [2]:

    
from layered_neural_network import LayeredNeuralNetwork
input_dimension = 2
lnn = LayeredNeuralNetwork(input_dimension=input_dimension)

Schooling

Let us send this young classifier to school to be taught functions like and, or and xor



In [3]:

    
from school.binary import Binary
Binary.teach_and(lnn)
Binary.teach_or(lnn)
Binary.teach_xor(lnn)









    



Teaching classifier task of and
Training for label and
Trained new Linear SVC with score 1.0
Teaching classifier task of or
Training for label or
Trained new Linear SVC with score 1.0
Teaching classifier task of xor
Training for label xor
Trained new Linear SVC with score 1.0






    Out[3]:





1.0

Looks like our LNN was a good student scoring perfect F1 score of 1.0 in each task.

Plotting weights

Let us plot the learnt weights. The weights will be of shape (classifiers_count, input_dimension + classifiers_count)



In [4]:

    
%pylab inline
weights = lnn.get_weights()
print(weights.shape)









    



Populating the interactive namespace from numpy and matplotlib
(3, 5)



In [5]:

    
plt.imshow(weights, cmap='gray', interpolation='none')
plt.title('Weights of LNN')
plt.xlabel('Inputs')
plt.ylabel('Classifiers')









    Out[5]:





<matplotlib.text.Text at 0x21d6cf914e0>

Understanding weights

The above shown matrix corresponds to the weights of classifier. Lighter shade indicates higher value, gray = 0, black = negative. Each row is a calssifier, with each column indicating the weight given to different input, previous classifier.

and classifier The first row shows AND classifier. Note the high value for input 0 and input 1 and the high bias. This is ON only when both inputs are high.

or classifier The second row shows OR classifier. Note the relatively low values for both inputs and bias.

xor classifier A linear classifier cannot learn XOR, but with the help of AND and OR classifier even a linear classifier can solve the dreaded XOR task. XOR -> big negative value for AND, positive value for OR.



In [ ]: